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BaCkcroUnd of t-hf» Tnvt>n^7r>n 

The present invention relates to digital image 
processing circuitry, which has utility~~in the inter- 
pre cation of electronic images generated, for example, 
from photographs or other forms of imaging in which it 
ir necessary to process electronic images in order to 
interpret or to extract the data contained in the 
laaga, e.g., in interpreting or extracting image data 
acquired by satellite photography or in interpreting or 
extracting image data in acquired search fingerprints 
or file fingerprints. 

In an automatic fingerprint identification system 
for matching search fingerprints to file fingerprints, 
of the kind described in Larcher et al- U.S. Patent No. 
4j_790, 564,. issued December 13, 1988 and entitled 
AUTOMATIC FINGERPRINT IDENTIFICATION SYSTEM INCLUDING 
PROCESSES AND APPARATUS FOR MATCHING FINGERPRINTS, the 
apparatus comprises a plurality of different subsystams 
communicating via a high-speed local area (LAN) 
network. Three of these subsystems are a ten print 
card input subsystem, a latent print input subsystem, 
and an encoding subsystem. The outputs of the print 
subsystems include analog or gray scale electronic 
Images of fingerprints, which are subjected to digital 
image processing and digital image filtering. One 
drawback of conventional digital image processing and 
.digital image filtering methods and apparatus is the 
large time overhead involved in performing the neces- 
sary image processing and image filtering computations. 

In some ca^es, the image quality of the gray scale 
image outputs is directly related to the quality of the 
respective print inputs to the two subsystems and those 



of ordinary sy.iU m ti.-e ert have- sought tc improve 
inagn quality by means of digital iin^ge processing and 
digital image filtering. 

The encoding subsystes: described in the '564 
patent includes appropriate circuit means for utilizing 
the electronic images co detect the minutiae in each 
fingerprint. Poor linage quality haf: an adverse effect 
on the process of detecting minutiae?. Poor image 
quality also has an adverse effect on correctly 
interpreting or extracting electronic image data 
acquired by satellite photography or other means. 

Hence, the capability of increasina the speed of 
making digital image filtering and digital image 
processing computations and the capability of improving 
the quality of digital images by means of digital image 
.iltertng and digital image processing are goals sought 
after by those of ordinary skill in the art. 

PbiectS of TnyontiT^p 

tlnn r ^" Object Of the present inven- 

tion to provide novel digital image processing and 
dig tal image filtering methods and apoaratus embodying 
high speed xmage processing computations. 

orov^r ^Z"^"^*^"" ^^'^--^ °f the present invention to 
provide novel digital image filtering methods and 
apparatus embodying high speed image filtering 
coBputations. 

p=,ovid! t" " 'T^" ""^"^ °' P"^^'" invention to 
2TltlrTl "^"""^ programed by . 

s^- l:: ::::::: rm ^^^^^^-^^-^-^^^ 

fingarprint cards ^c?-'- P'.="5r.ph,, 

vide new .„d i„proved methods and apparatus for 
genara..„g random access memory addresses. 
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application, e.g., in digital image processing and 
digital image filtering methods and apparatus. 

Other and further objects of the present invention 
will be apparent to those of ordinary skill in the art 
of digital image processing ^and digital image filtering 
from the detailed description set forth below. 

gy^Tnary of the Invention 

In accordance with one apparatus aspect of the 
present invention, there is provided in an electrical 
filter for filtering digital images, said filter having 
input means for receiving an electronic digital imag« 
having a boundary and output means for transmitting a 
filtered electronic digital image, the improvement 
characte2^zed by the combination of control processor 
means; and circuit means for filtering said received 
digital image, said circuit means incorporating at 
least one pipeline processing chain means for making a 
first plurality of digital image filtering 
computations, said chain means having input means to 
which at least one pixel of said received image is 
applied and output means from which said filtered 
digital image is coupled to said transmitting means, 
said processing chain means being responsive to a' 
second plurality of commands interpreted by said 
processor to enable said processing chain means to make 
predetermined ones of said first plurality of 
computations with respect to said pixel of said 
received digital image. 

In accordance with another apparatus aspect of the 
prasent. invention, there is provided in an electrical 
circuit for processing digital images having an input 
means for receiving at lease one electronic digital 
input Linage to be processed and output means for 
trcn3raitt:ing at xeast: one processed electronic digital 
oucpat: iaaqe, the improvement characterized by the 
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combination of control procassor means; video random 
accaas memory means; video random access memory 
controller means; and circuit means for processing at 
least one digital image^ said circuit means 
incorporating at least onA pipeline processing chain 
means for making a first plurality of digital image 
processing computations, said processing chain means 
having input means to which at least one pixel of said 
input image is applied and output means from which a 
processed input pixel is applied to said transmitting 
means, said processing chain means being responsive to 
a second plurality of commands interpreted by said 
processor to enable said processing chain means to make 
predetermined ones of said first plurality of 
computations with respect to said pixel of said input 
digital image. 

In accordance with another apparatus aspect of the 
present invention, there is provided in a address 
generator circuit for generating addresses in random 
access memory means for one or more pixels to be 
extracted from at least one Electronic digital input 
image having a boundary, said extracted pixels 
thereafter being processed to derive an electronic 
digital output image, the improvement cha racterized by 
the combination of means for deriving from said input 
image at least one logical electronic digital image 
located at or within said boundary of said input image, 
said logical image having at least one pixel and having 
an arbitrary origin cf Cartesian coordinates; means for 
identifying at least one pixel to be extracted from 
3aid logical image; aeans for scanning said logical 
imaga in line and column directions to generate a pair 
of ntemory storage addresses, each pair comprising a row 
storage address and a column storage address in said 
Tiemory for each of said identified pixels; means for 
extracting said identified pixel; nieans for processing 



eiach of said extractied pixels by pipeline processing 
means thereby to obtain a processed output pixel; and 
means for simultaneously generating a pair of memory 
storage addresses for each processed pixel, each pair 
comprising a row storage address and a column storage 
address in random access memory means, and storing said 
processed pixel in said memory at said generated pair 
of addresses . 

In accordance with another apparatus aspect of the 
present invention, there are provided random access 
memory means in a digital filter circuit and in a 
digital image processing circuit characterized by a 
memory organization comprising at least four memory 
sections, each section being arranged in 4-bit size 
planes, whereby said memory can store 4-bit, 8-bit, 12- 
bit, and 16-bit wide digital images; ^nd_random access 
memory means in a look-up table for a digital image 
filter circuit and for a digital image processing 
circuit characterized in that said table's lut data 
comprise numerical luts and/or binary luts and in that 
said memory is organized into at least two banks, each 
bank for storing numerical lut data or binary lut data; 
x«ZKi«and»«j^««acx»«XW«x»eft«»x;4axto 

xi«wjfXjAttta.«««xjtext*>«Ju»«^ 
'^a't!ca:ea«textvocxb«o|g«>x»«akxi30aJu;efi^ 

In accordance with a method aspect of the present 
invention, there is provided in a method for processing 
at least one electronic digital input image having a 
boundary to derive a processed electronic digital 
output image, the improvement characterized by the 
steps of deriving from said input image at least one 
logical electronic digital image located at or within 
the boundary of said input image, said logical image . 
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having at least one pixel and havinq an arbitrary 
cric^in of Cartesian coordinates; identifying at least 
one pixel to be extracted fron said logical image; 
scanning said logical image in line and column 
directions to generate a pair of memcry storage 
addresses in random access memory, each. pair comprising 
a row storage address and a column storage address in 
said mem.ory for each of said identified pixels; 
extracting and processing each of said identified 
pixels by pipeline processing coiriputations to obtain a 
processed output pixel; and simultaneously generating a 
pair of memory storage a ^dresses for each processed 
pixel, each pair comprising a row storage address and a 
column storage address in random access memosry, and 
storing said processed pixel in said memory at said 
generated pair of addresses. 

In accordance with another method aspect of the 
present invention, there is provided in a method for 
generating addresses for random access memory for one 
or more pixels to be extracted from at least one 
electronic digital input image having a boundary, said 
extracted pixels thereafter being processed to derive 
an electronic digital output image, the improvement 

steps of deriving from said input 
image at least one logical electronic digital image 
located at or within the boundary of said input image, 
said logical image having at least one pixel and having 
an arbitrary origin of Cartesian coordinates; 
identifying at least one pixel to be extracted from 
said logical image; scanning said logical image in line 
and| column directions to generate a pair of memory 
storage addresses, each pair comprising a row storage 
address and a column storage address in said memory for 
each of said identified pixels; extracting and 
processing each of said identified pixels by pipeline 
processing computations to obtain a processed output 
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pixel/ and simultaneously generating a pair of memory 
storage addresses for each processed pixel, each pair 
comprising a row storage address and a coluijin storage 
address in random access memory, and storing said 
processed pixel in said memory at said generated pair 
of addresses. 

FIG. lA is a block diagram depicting tha ganaral 
assignment of memory locations in tha video RAM 23, 
Image Storage (Imo-Imj) RAM 32, and LUT RAM 372 in a 
preferred embodiment of a general purpose digital image 
processing circuit 30 constructed and operated in 
accordance with the present invention; 

FIG. IB is a block diagram illustrating an exampla 
of one specific use of the general purpose digital 
image processing circuit 30 constructed and operated in 
accordance with the present invention; 

FIG. 2A is a block diagram showing the general 
aspect of a host computer system 10 operating with a 
general purpose digital image processing subsystem 30, 
which incorporates a general purpose digital image 
processing circuit 30, the subsystem 20 and circuit 30 
being constructed and operated in accordance with the 
present Invention; 

FIG. 2B is a block diagram showing a preferred 
IbiBbodiment of said processing circuit 30; 

FIG. 2C is a block diagram showing details of the 
interface 14 also described with reference to FIG. 2A; 

FIG. 3 is a block diagram showing the general 
aspect of a video ham controller ("VRAMC-) 22 also 
described with rafarenca to FIG. 28; 

FIG. 4 is a block diagram showing the general 
aspect of a crossbar unit ("CRX") 31 incorporated in 
said processing circuit 30 also described with 
rafarenca to FIG. zb; 
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riG. 4A is a block diagram showing an exampla of 
specific functions performed by CRX 31; 

FIG. 5 is a block diagram showing the general 
aspect of an address generator ("ADD GEN") 3 3 
incorporated in said processing circuit 30 and also 
described with reference to FIG. 2B, said generator. 33 
being constructed and operated in accordance with the 
present invention; 

FIG. 5A is a diagram illustrating image scan or 
scanning directions; 

FIG. 5B is a diagram illustrating coordinates of 
an electronic image composed of six pixels in a 2 x 3 
array; 

FIG. 5B' is a timing diagram pertinent to the 
operation" of address generator 33; 

FIGS. 5C and 5C' are diagrams illustrating 
pertinent aspects of pixel extraction performed by 
address generator 33; 

FIG. 5C'' is a timing diagram illustrating pixel 
extraction at HPIX clock rate; 

FIGS. SD and 5D^ are block diagrams showing the 
general aspect of ALUX 331 and ALUY 33 2. incorporated 
in address generator 3 3 described with reference to 
FIG. 5; 

FIG. 5E is a block diagram showing pertinent 
aspects of the synchronization by address generator 3 3 
of certain components of the digital image processing 
circuit 30 described with reference to FIG. 2B; 

FIG. 5F is a block diagram showing the general 
aspect of the exchange of image information between 
Image Storage RAM 3 2 and VRAM 23, under the control of 
address generator 33; 

FIG, 5G is a block dieigram showing the general 
aspect of the writing of image data to Image Storage 
RAM 3 2 by address generator 33; 



FIG. 5G' is a simplified blocJ: diagram showing the 
arrsngement of Image Storage RAM 32 into four 4-bit 
deep planes; 

FIG. 5H is a timing diagram pertinent to image 
scanning by address generator 33; 

FIG. 51 is a diagram illustrating the relationship 
between a physical image and a logical image and the 
use of ghost pixels to define image borders; 

FIG. 5 J is a timina diagram illustrating the way 
ROW and COLUMN addresses are incremented in a normal 
scanning of an n lines bv m pixels image; 

FIG. 6 is a block diagram showing the general 
aspect of a convolution unit ("CONVOL") 34 incorporated 
-in said processing circuit 3 0 also described with 
reference to FIG. 2B; 

FIG. 6A illustrates the interconnection of compo- 
nents of CONVOL unit 34 in performing identity and 
linear combination functions with respect to two images 
(A, B) and two constants (a, 0) j 

FIG. 6B illustrates the interconnection of compo- 
nents Of CONVOL unit 34 in performing the function of 
multiplication. of two images (A, B) ; 

FIG. 6C illustrates the interconnection of compo- 
nents of CONVOL unit 34 in performing the function of 
convolution on a two-dimensional kernel of up to 3 x 4 
pixels ; 

FIG. 6D illustrates the interconnection of compo- 
nents Of CONVOL unit 34 in perfor ing the function of 
convolution on a one-dimensional kernel of ud to 1 x 16 
pixels; 

FIG. 6E illustrates the interconnection of compo- 
nents Of com^'OL unit 3 4 in performing the function of 
convolution on a one-dimensional kernel of up to 1 x 32 
pixels; *^ " X X 



/o 



I 0 1 o 't 

FIG- 6F is a block diagram shov/.tng tna general 
aspect of morphology and convolution control ("HORCON") 
unit 3410 incorporated in CO^fVOL unit 34; 

FIG. 60 is a block diagram showing the intercon- 
nection of certain components of VFJ?. chips 3411 and 
3412 during convolution of a i x 16 kernel; 

FIG. 6H is a block diagram showing interconnection 
of certain components of VFIR 3 411 and 3 412 during 
convolution of a 3 x 4 kernel; 

FIG. 61 Is a block diagram showing the intercon- 
nection of certain compo!'ents of VFxR chips 3411 and 
3412 during convolution of a l x 32 kernel; 

FIG. 6J is a simplified block diagram of a 3416 
circuit embedded in 3^11 and 3412 VFIR chips; 

FIG. 6K is a block diagram showing the general 
aspect of delay lines 3413, 3414 incorporated in CONVOL 
unit 34; 

FIG. 6L is a timing diagram pertinent to explana- 
tion of the operation of delay lines 3413, 3414; 

FIG. 6K is a timing diagram illustrating the 
computation of a 3 x 4 kernel convolution under CT0 and 
CTl signals; 

FIG. 6M" is a timing diagram illustrating the 
computation of a l x 16 kernel convolution under CT0 
and CTl signals; 

FIG. 6M"' is a timing diagram illustrating the 
computation of a i x 32 kernel convolution under CT0 
and CTl signals; 

FIG. 6N is a timing diagram illustrating the 
behavior of the CT0 and CTl signals with respect to the ' 
various computations made by cONVOL unit 34; 

FIG. 7 is a block diagram showing the general 
aspect Of a logic ("LU-) unit 35 incorporated in said 
processing circuit 30 also described with reference to 
FIG, 2B; 
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FIG. 7A is a block diagram illustrating the func- 
tion of data scaling performed by logic unit 35; 

FIG. 7B. is a block diagram showing the general 
aspect of the MIN/MAX detector 357 described with, 
reference to FIG. 7; 

FIG. 8A is a block diagram showing the general 
aspect of a morphology ("MORPH") unit 36 incorporated 
in said processing circuit 30 also described with 
reference to FIG. 2B and showing pertinent data and 
synchronization signal paths for operation of the PIMM 
circuits 361, 362 in parallel mode; 

FIG. 8B is a block diagram similar to FIG. 8A 
showing pertinent data and synchronization signal paths 
for operation of the PIMM circuits 361, 362 in serial 
mode ; 

FIG. 9 is a block diagram showing the general 
aspect of look-up table (••LUT") unit 3 7 incorporated in 
said processing circuit 3 0 also described with refer- 
ence to FIG. 2B; 

FIG. 9A is a block diagram illustrating the data 
and address signal paths that obtain during initializa- 
tion of the LUT RAM 372; 

FIG. 9E is a block diagram illustrating the data 
and address signal paths that obtain during processing 
of 16-bit numerical images by LUT 37; 

FIG. 9C is a block diagram of LUT 37 illustrating 
the data and address signal paths that obtain during 
processing of e-bit binary images with PIMM circuits 
361, 362 of morphological unit 3 6 operated in aerial 
mode; 

FIG. 9D is a block diagram of LUT 37 illustrating 
the data and address signal paths that obtain during 
processing of cwo 8-bit binary images with the PIMM 
circuits 361, 362 of morphological unit 36 operated in 
parallel mode; 



rIC. 9E Is a block diagram of LUT 37 illustrating 
the data and address signal paths that obtain during 
processing of an identity function on a 16-bit ^ 
numerical inage using a binary lut, where the eight 
most significant bits ("MSB") are by-passed through LUT 
circuit 371; 

FIG. 9F is a simplified block diagram pertinent to 
the description of the circuitry of FIG. 9A, which 
illU3trates signal paths during the initialization of 
the. LUT RAM 372; 

FIGS. 9G, 9H are simplified block diagrams perti- 
nent to the description of the circuitry of FIG. 9B^ 
which illustrates processing of 16-bit ntuaerical 
Images; 

FIG. '"91 is a simplified block diagram pertinent to 
the description of the circuitry of FIG. 9C, which 
illustrates processing of an 8-bit binary image with 
the PIMM circuits 361, 362 operated in serial mode; 

FIG. 9 J is a simplified block diagram pertinent to 
the description of the circuitry of FIG. 90^ which 
illustrates processing of two 8-bit binary images with 
the PIMM circuits 361, 362 operated in parallel mode; 

FIG. 9K is a simplified block diagram pertinent to 
the description of the circuitry of FIG. 9K, which 
illustrates the processing of an identity function; 

FIG. 9L depicts the general assignment of memory 
locations of LUT RAM 372 when working in mixed mode, 
i.e., one numerical lut and 256 binary luts; 

FIG. 9M depicts the general assignment of memory 
locations of LUT RAM 372 as loaded when processing a 
load- lut. command; 

FIGS. lOA, lOB are timing diagrams useful to 
explain the opeiration of the general purpose digital 
image processing circuit: 3 0 constructed and operated in 
accord^ince wiLh the present invention; 
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FIC 11 is a block diagram depicting an example of 
roicrocode for the operation of the general purpose 
digical image processing circuit 3 0 constructed and 
operated in accordance with the present invention. 

Detailed HescriPtion 
1. SV8tein Viewpoint 

Referring to FIG. 2A, the general purp'ose 
electronic digital image processing circuitry 30 
constructed and operated in accordance with the present 
invention can be incorporated in a general purpose 
electronic digital imago processing subsystem 20. One 
example of intended use for subsystem 20 is in an 
encoding subsystem of the kind described in the above- 
. identified '564 patent. 

The subsystem 20 is connected via cable 10a to 
interface circuitry 14. The purpose of interface 
circuitry 14 is to enable the subsystem 20 to be 
compatible with the architecture of the host computer 
10, which can embody, for example, conventional 
MicroChannel Architecture f -MCA" architecture) . The 
interface 14 includes a control register 317, and the 
subsystem 20 includes a status register 318. The host 
computer 10 additionally incorporates a central 
processing unit ("CPU") 12, RAM 13, and peripherals 
(not illustrated) . 

Referring to FIG. 2C, the interface circuitry 14 
includes circuitry for connecting Data ( "MCA-DATA" ) , 
control ("MCA-CTRL"), and Address ( "MCA-ADDIIESS") 
signals from the MCA bus to the circuitry 30 in a Slave 
mode. To perform this function, the interface 
circuitry 14 comprises a programmable ROM ("PROM") 141, 
which operates on power-up of the host computer 10 to 
initialize a standard FPGA chip (MCA XILJNX chip) 142. 
In operation, chip 142 sends requests for data to the 
VRAM controller 22 and receives acknowledgement of 



thosR requests. Under the control of "enable" and 
"clock" signals from the FPGA chip 142, a conventional 
three state output register 144 provides address 
information from the MCR. bus via circuit 145 to VRAM 
23. Three states are required to account for the 
competing requests for addresses by the V7iAK controller 
22, the control processor 21 (FIG. 2A) , and the host 
computer 10 via the ch.-i.p 142. Control signals from the 
MCA bus schedule the transfers between host computer 
RAM memory 13 and video RAM 23. Data signals from the 
MCA bus are directly coupled via the transceivers 143 
and the DATA line to the control processor 21 and the 
video RAM 23. Eight LSB bits from the MCA Data bus are 
used to configure internal Programmable Option Set 
registers (defined in MCA architecture) in FPGA chip 
142 at system initialization and to provide control via 
the read/write control register 317 of the host 
computer 10 (FIG. 2A) . 

In accordance with the in/ention, the subsystem 20 
(FIG. 2A) incorporates a digital signal processor 21 
acting as a control processor ("CP"), a video RAM 
controller 22, video RAM 23, and a general purpose 
digital image processing circuit 3 0 constructed and 
arranged for operation in accordance with the present 
invention. 

The generaL aspect of video RAM controller 22 is 
depicted in FIG. 3. Referring to FIG. 3, the 
controller 22 incorporates an address generator 224 
comprising X and Y counters for generating VRAM 
addresses. Block 221 provides RAS, CAS, Refresh. and 
States signals for video RAM 23 on the primary bus or 
P_Bus (FIG. 2A). Exchange control block 222 provides 
synchronization signals to control the timing of data 
exchange on S_Bus. Arbiter block 223 controls the flow 
of data on the primary bus (P_Bu6) in response to 



- requests from the host CPU 12 and th« CP 21, and in -< 
responsd to requests for Refresh and Exchange signals. 

In general, the control register 317 (FIG. 2A) In 
conjunction with the status register 318 functions to 
synchronize the operation of host computer 10 and the 
image processing subsystem 20. Upon power up the 
subsystem 20 is placed In a reset state so that th# 
host configuration system process can run. The VRAM 
controller Is initialized* Program code and data «re 
loaded into VRAM 23 (FIG. lA) , e.g., object code and 
data for control processor 21, object code and data 
(commands CMO 1 to CMD N) for the operation of circuit 
30; additionally, loo)c-up table data are loaded into 
Che LUT RAM 372. All the programmable chips aria 
programmed and initialized with default values. 
Following initialization, the control processor 21 
proceeds through an idle loop (FIG. 11) in which it 
repeatedly reads the status register 318 (FIG. 2A) 
looking for a GO signal asserted hy the host 
computer 10 (FIG. lOB) • 

The control register 317 is activated to run the 
CMD programs, i.e», CMD 1 to CMD N (Flg^. lA) , stored in 
the VRAM 23. When the programs have rxin, a statue, flag 
is set and informs the host computer 10 (FIG. 2 A) that 
a processed image is stored in VRAM 23 (FIG. 2B) . The 
host computer 10 copies that processed image to RAN 13,. 
for subsequent use, e.g., for minutiae extraction when 
the subsystem 20 is used in conjunction with the 
fingerprint identification system of the aforesaid '564 
patent. 

In accordance with the present invention, 
electronic images can be stored anywhere in VRAM 23 and 
Image Storage RAM 32. 

FIG. IB snows a specific example of a generalized 
process performed by the general purpose digital image 
processing circuit 30 (FIG. 28) from the time an 
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electronic digital image 101, regardless of its source 
and data content, is acquired by the host coinputelr 10 
and stored in VRAM 23 until the image processing 
functions are performed and result in production of a 
processed electronic digital image 106, which is copied 
by host computer 10 to RAM 13 for subsequent, 
processing. In accordance with the invention, the 
circuit 3 0 is operated by the CMD programs stored in 
VRAM .2 3 and provides digital image filtering and 
processing functions, as many times as required by the 
user, with respect to gray scale digital images and/or 
binary digital images, to obtain the desired processed 
digital image, which is copied from Image Storage RAM 
3 2 to VRAM 23 and thence to host computer RAM 13 for 
further processing. 

It is to be understood that other specific 
examples of a generalized process performed by the 
general purpose digital image processing circuit 30 
Include (1) repetitive digital image filtering of a 
gray scale image as indicated by the feedback loop on 

A- »- . 

image 103 (FIG. IB) to produce a processed digital 
image available to the host computer; (2) repetitive 
digital image filtering of a converted image 104, as 
Indicated by its feedback loop, to produce a processed 
digital image available to the host computer; 
(3) repetitive digital image filtering of binary Images 
105 and 106 as indicated by their feedback loops (FIG. 
IB) to produce a processed digital image available to 
the host computer; ard (4) other combinations of 
feedback looping (not de|)icted) to obtain repetitive 
filtering and processing jof whichever digital images in 
whatever combinations the user selects, to produce a 
processed digital image available to the host computer. 

The host computer 10 requests the subsystem 2 0 to 
operate by setting the GO signal in the register 317 
(FIGS, lOB, 11) . In idle state, the control processor 
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21 polls the status register 318 while waiting for the 
GO signal to start an image processing computation 
task. 

An inage processing computation task is a sequence 
of connands (CUD) ending with a STOP CODE (hexadecimal 
AOOCOOOO) . Each command is made up of 32'-blt words, 
the left-most 16 bits or most significant bits ("MSB") 
of which constitute the address of an internal register 
incorporated in unit 30 and the right -most 16 bits or 
least significant bits ("LSB") of which constitute data 
to put into that register. Each command ends with a 
START CODE (hexadecimal 80000000) . 

Following initialization, the control processor 21 
proceeds through an idle loop (FIG. 11) in which it 
repeatedly reads the status register 318 (FIG. 2A) 
looking for • GO signal asserted by the host computer 
10 (FIG. lOB) . A GO signal is asserted when an 
electronic digital image is acquired by the host 
computer 10, which copies that image to VRAM 23 and 
•ctivatss the control register 317 and sets the statu, 
register 318 (FIG. 2A) . 

Opon assertion of a go signal, the control 
processor 21 read, and interprets the command. CMD 1- 
CUD K (FIG. lA) and goe. through one or mor« 
programmation loop, and one or more processing loops 
(FIG. 11) . ' 

^ ^ in particular, during the elapsed time interval 
^ (FIG. lOB) between assertion of the GO and MANIP 
•Ignals, all of the dedicated internal register, in 
circuit 30 are programmed. After control processor 21 
csserts the manip .ignal to start a processing loop, 
Address generator 33 begins to assert synchronization ' 
sxgnals (FIG. lOB, . At the end of elapsed time 
interval -e" (fig. lOB) the synchronization signal, are 
asserted by th. last stage of the processing circuit 
^o, i.e.. LUT 37, the time incerval "e" being 
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representative of computational tine delay in 

circuit 30. The control processor 21 reasserts the 

MANir signal depending on whether further image data 

are to be processed, as indicated by the presence or 

absence of image synchronization signals produced by 

the address generator (FIG. lOB) . 

When a stop code, hexadecimal AOOOOOOO (FIG. lA) 

is reached, the control processor 21 asserts the EXE 

signal (FIG. lOB) to end the processing cycle and to 

reenter the idle loop. 

2. General Purpose Digital 

Image Procaaaing circuit ' 

The preferred embodiment of general purpose 
digi t al image Pfocessing circuitry 30 constructed and 
opera t'^d in accordance with the present invention is 
depicted in block diagram form in FIG. 23. The 
circuitry 30 includes crossbar ("CRX") unit 3I, Image 
Storage RAM 32, address generator ("ADD GEN") 33, 
coiivolution ("CONVOL") unit 3^^ogic ("LU") milt 35^ 
morphological J-MORPH-) unit 36^ and look-up tabir 
("LUT") unit 37. Sufficient memory is provided in RAM 
32 to provide four sections (32A-320) of memory, Im,- 
Im,, separately to store simultaneously four 512 x 512 
16 bit electronic Images (Imo to Im,) which can be gray 
scale and/or binary Images. 

In operation of circuit 30, RAM 32 can simultane- 
ously store an acquired electronic digital image 
transmitted for processing by host computer 10 (Im©) , 
two images (Im,, Im,) at various intermediate stages of 
processing by the circuitry 30, and a processed digital 
image for transmittal to the host computer (Im,) . A 
separate address geuerator 33 (four in total number) la 
provided for each section (32A-32D) of Image Storage 
RAM 32, i.e., generators 33A-33D. 

In accordance with the present invention, the 
proces-sing circuitry 30 embodies pipeline erchxtecture. 




Vh:*ch advantageously enable:^i the greatest number of 
processing operations and computations to be performed 
yithin the shortest possible overhead time. Other 
interconnections and operation of components of the 
^ind embodied in circuit 30 are not nearly as efficient 
aa th« pipeline architecture embodied in circuitry 30. 

Parallel processing as embodied in circuitry 3 0 
means that the following operations can be performed at 
the same time, e.g., transfer of an image between VRAM , 
?3 and RAM 32 and processing of a previously 
transferred image in circuit components 34, 35, 36, 37, 
can be performed at the same tim^. 

The address generator 33 constructed, and operated 
in accordance with the present invention plays a key 
role in the. parallel processing performed by circuitry 
30, as is described elsewhere herein, in that computa-- 
tion and exchange-of-data (transfer) modes run at the 
same time, - ' 

' A. croggbar Unit 31 

The functions and purposes of the crossbar unit 
(**CRX**) 31, whose general aspect is depicted in FIGS* 
4, 4A/ include the provision of read/write comraunica-* 
tion between video RAM 23 and Image Storage RAM 32 via 
s_bus, the placement of images stored in Image Storage 
RAM 32 via lines Iq-I, upon A_Bus and/or B_Bue for 
transmittal to CONVOL unit 34, and the connection of 
processed images received from LUT unit 37 via R_Bu» 
for transmittal to Image Storage RAM 32 via lines lo-Ij. 

Referring to FIG. 4A, the CRX unit 31 includes two 
programmabJe registers, i.e., control register 3171 and 
constant value register 3181. The control register 
3171 is programmed to determine which buses, i.e., 
A_Bus, B_Bus, S_Bus and R_Bus, are connected to which 
image lines, ..a., im^, im, , Imj, and Ira,, of the Image 
Storage RAM 32. In the example depicted in FIG. 4A, 
A_Bus, B_Bus, and R_Bus are connected respectively to 




, 16-bit ima^s lines I^^ It, and I), while S_Bus id 
cohnacted to 16rbit image line Zj. 

Referring to FIG. 4/ bloOcs 311, 312, and 313 
depict respectively the operand connection for 16-bit 
iaages on image lines lo-Ij* Block 314 depicts the 
operand connection of 16-bit images via S_Bus on images 
lines lo-I}' Register 3181 is initialized to provide . 
both the. definition of the border of an image and also 
to aett a constant , pixel value on B^Bus. 

synchronization for operating CRX unit 31 is 
provided by unit 3 IS under the control of synchroni- 
sation signals, s, C^.n-input) , provided by address 
generator 33. CRX synchronization unit 315 couples 
synchronization signals, So (V-«wtput) , to the CONVOL 
unit 34. - • , 

B. Address Qenera^^ar 33 

In order to process an acquired image stored, for 
example, as image Imo in Image Storage RAM 32 (PIC. 2B) , 
it is necessary to extract pixels from the image Im^ 
("source image") and produce an address in Image Im,, 
Imi or Ims to which to write the pixels processed by the 
processing chain comprising units 34, 33, 36, and 37 
("result image"). The address generator 33 performm 
the address generation for pixel extraction and pixel, 
writing functions. As noted elsewhere herein, one 
address generator 33 for a total of fouc, i.*. , 23A, 
33B, 33C, and 33D, is provided for each Image Storage 
RAM 32 section 32A-32D, i.e., in^, la,, in, and Im,. 

Each addrasB generator unit 33 (FIG. 5) has three 
poesible modes of operation, i.«,, "Master" mode 
(TIG. SE), "Slave" mode (FIG- 5E) , and "Exchange" mode 
(FIG. 5F) . A generator 33 cannot simultaneously ' 
operate in Masc^r and Slave modes, or in Exchange and 
any other node. 



Addresses for pixels to be processed, i.e., source 
image pixels, are generated by an address generator 33 
operating in Master mode, e.g», generator 3 3A (FIG. 
5E) . Addresses for pixels that have been processed by 
units 34, 35, 36, 37, i.e., result image pixels, are 
generated by an address generator 3 3 operating in Slave 
mode, e.g. , generator 3 3B (FIG. 5E) . As addresses are 
generated by generator 3 3B, the processed pixel data 
are written to Image Storage RAM by Write unit 341 
(FIG. 5G) of generator 33B. Synchronization unit 338 
(FIG. 5) of generator 33A (FIG. 5E) provides horizontal 
and vertical synchronization signals (hereinafter H^, 
V^) for the processing chain, units 34, 35, 36, 37. 
Unit 37 provides horizontal and vertical 
synchronization signals (V^ and H^) for generator 33B 
(FIG. 5E) and its Write unit (FIG. 5G) . 

In Exchange node (FIG. 5F) , an address generator 
33 causes pixels to be written to Image Storage RAM 32, 
e.g., memory Imo, from VRAM 23 via CRX unit 31 or causes 
pixels to be written to VRAM 23 from Image Storage RAM 
32, e.g., memory Imj, via CRX unit 31. Synchronization 
signals H,, V, are provided by generator 33. Direction 
control and request to read/write sicrnals are provided 
by generator 33 and VRAM controller 22. 

Address generation involves an image scan or 
scanning process; eight image scanning directions or 
scan modes from any one of four arbitrary origins of 
coordinates, e.g., Xo. Yo, XoS Yo'/ V, Yo*; Yo", can 

be produced by address generator 33 (FIG. 5A) . 
Coordinate systeti origins can be located anywhere. 
Initial values for the coordinate system origin are 
stored during initialization in X^, Y^ registers at 335, 
337 (FIG. 5) . 

Images can be scanned to extract various numbers 
of pixels, e.g., ail pixels (PIGS. 5B, 5B'), one or 
more pixeJs out of n pixels, every other pixel (FIGS. 
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3C, 5C\ 5C'*) , i.e., a subsanipllng process to decrease 
the aire of an electronic image, or by oversampling to 
increase the image size. In the oversampling process 
of the present invention, a series of pixels P^, Pj^ Pj, 
. . • , P^ is sampled in such manner that the resultant 
image is a series of pixels, e.g., Pq, Po/ ^of Pi/ 

P:/ Pj/ Pi#^ ; Po/ P./ Par i.e., a gap of two pixels 

width has been inserted after each source pixel. The 
size of the gap can be varied to any desired width up 
to 16. In the oversampling process the address 
generator Slave mode is used to increment the address 
by t^e amount of the size of the gap selected. 

The scanning process and the address generating 
procers involve implementation of a kind of counter in 
the generator 33. The counter is constructed and 
implemented to increment or decrement by any desired 
amount less than 16. The origin coordinates (X^, Yo) 
and t-he amount of incrementation or decrementation (IX^, 
lY^) arc set during initialization or during the running 
of certain CMD commands* 

In Master mode, the synchro unit 338 (FIG. 5) of 
generator 33 (1) generates H. and V, synchronization 
signals and (2) generates the size of a border to 
sxirround the image (hereinafter from time-to-time 
referred to as ••ghost pixels"). In Slave mode, the 
synchro unit 338 performs one function in relation to 
H^/V^ signals, enables, i.e., unit 338 generates 
destination addresses for processed pixels. Only 
-good" processed pixels, not ghost pixels, are written 
to the Image Storage PAM 3 2 during the Slave mode of 
operation. In accordance with the invention, the Write 
unit 341 (FIG.^ 5G) operates only when H„ and V„ are 
high, ;:hus preventing the writing of ghost pixels, 
w.iich occur only when is lov (see waveforms in 
FIG. 55')- 



In conventional digital intagd procdaaing 
elrcilltty, the selection of kernels for processing is 
1 ir.it ad . to kernels located completely vinsJtda the 
phyaleal image border. In accordance: wit*i.th«"preaent 
invention,, the uae of logical iinagea and ghoat pixel 
bonders enablea the user of circuit 3.0 to select 
kernels (PIG. 51), e.g., pixel p^, cloaa to the border 
of the phyaical inage by adjusting the aite of the 
ghoat pixel border surrounding the logical image. 
Additionally, by surrounding the phyaical image border 
or boundary with a ghost pixel border of any desired 
width dependent upon the shape of thi« Xarnel, e.g. , one 
pixel wide aa ia depicted in PZG. ilJ, logical images 
right at the phyaical image border, a.g*, image P^' 
(PIO. 51) , can be readily proeeaaed. 

Xn the acanning proeeaa performed by addreaa 
generator 33, ghoat pixela are produced during the 
flybaeJc interval. At, which followa the acan of a line 
of pixela in the logical image (FIG. 5H) . The aize of 
the ghoat pixel border can be adjuated, for logical 
Imagea well inaide the phyaical image border, by 
adjuating the flyback interval. Thereby, the circuit 
30 both ailowa the making of certain computations on 
kernela that include ghoat pixela, which are hot other- 
wiae poaaibxe in conventional practice, and alao ailowa 
the user of circuit 30 properly to construe the results 
of such coaputations. 

The size of an image line, i.e., the number of 
pixels in a line, is programmed initially in the 
register 33i of the Scan (X) Counter 334. The size of 
an image, i.e., the number of lines, is programmed 
initially into the register 337 of the Scan (Y) 
Counter 336. 

X and Y address data are provided by operation of 
ALU X unit 3 31, ALU Y unit 3 32, and ALU Control 333 
(FIG. 5). Initial Xo and IX, data are* programmed into 



r*giat«r9 331a, 331b (FIG. 5) . SAnllariy, Initial Yq 
and IYq data ar« programned into registers 332a, 332b 
(PIO. 5) . 

FIC. 5B shows an example of a 2 x 3 pixel logical 
iaage, where each pixel comprises 16 bits, located in i 
coordinate system of origin Xq, Yq. Image scanning 
is performed in the positive X-direction (FIG. 5A, 
SCAN MODE - 0) . FIG. 5B' depicts a timing diagram 
showing the extraction of all pixels in the logical 
image depicted in FIG. 5B. FIGS. 5C, 5C', and 5C" 
also depict image scanning in the positive X-direction 
of a six pixel linear image from which every other 
pixel is extracted by the scanning process. 

riQm, 5D and 5D' depict the general aspect of the 
counters," AliU X tinit 331 and ALU Y xinit 332 (FIG. 5). 
Referring to FIG. 5D, at the start of scanning the Xq 
value is loaded into the X ALU register 3313 through 
the multiplexer 3312, providing the first X address. 
For the next cycle (i.e., the next pixel or the next 
line depending on the way the image is scanned) , the X 
value is added with the increment value IX through the 
add/ subtract circuit 3311 to produce the X + IX address 
which is fed back to the 3311 circuit. The increment- 
ing process is continued until the end of the line 
or the end of the image depending on the scanning. 
FIC. 50' is identical to FIG. 5D and performs the same 
process to calculate the desired address Y-values. 

In suamary, given an n x m pixel image containing 
n lines (rows) and a columns (m pixels per line), it Is. 
necBs.ary for the incrementing countars depicted in 
FIGS. SD, 5D' to produce n line addresses and n x m 
colu»n addresses (FIG. 5J) . In operation, the ALU 
control unit 333 (FIG. 5) directs the ALU X unit 331 
and ALU Y unit 33 3 when to compute new X„, Y„ addresses. 



Pertinent timing of functions performed by the 
generator 33 is shown in FIG. 5B'. The rising edge of 
the HPIX clock signal indicates the start of a new 
pixel cycle. At the same time that the first pixel, P^, 
is read from the storage image, the synchronization 
signals H, (Horizontal synchro) and V, (Vertical 
synchro) go simultaneously high to indicate the 
beginning of the image, i.e., the first pixel of the 
first line. 

H, remains high until the end of the image. Vs 
goes low at the end of each line and remains low until 
the border or ghost pixels, which attribute is 
determined by convolution requirements and initialized 
in RX register 340 (FIG. 5) , have gone. 

In effect, the start of the row signal, V,, is 
delayed by the time corresponding to the width of the 
ghost pixel border selected. Assertion of H^, v. 
signals is delayed by the amount of time corresponding 
to the time elapsed ("computational delay") in computa- 
tion or processing of the input or source image pixels 
Po to Pj by the processing chain units 34,. 35, 36, 37 to 
form processed or result image pixels p^' to P,'. 

In accordance with present invention, the timing 
of functions performed by address generator 33 is bised 
upon the processing of logical, not physical, images. 
The following advantages are thereby obtained: 

(1) the time to complete processing functions is 
shorter when timing is related to logical 
images that are smaller thwn physical images; 

(2) the ability to elect to scan in one of up to 
eight scan modes or directions, e.g., to 
reverse video, shortens the time of certain- 
computations performed by circuit 30; 

(3) the ability to elect to scan in one of up to 
eight scan modes or directions allows the use 
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of vertical linear filters created by 
exchanging pixel rows and columns ; 
(4) the ability to elect to scan in one of up to 
eight scan modes or directions allows the use 
of the oversarcpling technique described 
elsewhere herein, which itself is a new 
feature in digital image processing, to 
enable certain computations to be made that 
could not other -^ise be made; and 
(5) the ability to elect to scan in one of up to 
eight scan modes or directions enables 
addresses to be computed by address generator 
33 on the fly fcr each elemental process 
performed by the processing chain of digital 
image processing circuit 30. 

C. Image s torage R/^y* -^-t 
Image storage RAM means 32 is comprised of SRAM 
components arranged in a novel configuration in 
accordance with the present invention. In accordance 
with the present invention, RAM 32 comprises four 
memory sections 32A, 32B, 32C, 32D (FIG. 2B) each 
having sufficient capacity to store a full electronic 
xmage (la,, Im., im,, i„^, of the largest size expected 
in the operation of the subsystem 20 (FIG. 2A) con- 
structed and operated in accordance with the present " 
Invention. ^ 

Each memory section 32A, 32B, 32C, 32D can contain 
the same or different amounts of total memory. m the 
preferred embodiment, the four memory sections are 

: arra,^ged in four 4- 

bit or nybble size planes ^FIG srM i 
„^ . . ' H^anes (txG. 5G' ) and each memory 

T'Ts T\" ^"=°--^°"*^^ - " «-c..e.s generator 33 



As noted elsewhere herein, with reference to TIG, 
5T, electronic images are transferred to and from VRAM 
23 from and to Image Storage RAM 32. Syncjtjjpalzatlon 
signals are required when writing an image fi^om a VRAM 
23 to the Image Storage RAM 32. The synchronization 
signal, i.e., "Request To Exchange" (FIG. SF) , from the 
VRAM controller 22 is an exchange request, i.e., a 
irequent to exchange data between VRAM 23 and Image 
Storage RAM 32. The address generator 33 then produces 
synchronisation signals H„ V„ so that the VRAM 
controller knows when to produce good addresses. 

D. Convolution Unit 3A 
Central to the digital signal image processing 
functions performed by circuit 30 (FIG. 2B) are the 
digital signal filtering functions performable as a 
result of computations made rapidly and repetitively at 
high rates by circuits incorporated in convolution 
iuiit 34. 

The general aspect of convolution unit 34 is 
depicted in FIG. 6. Convolution unit 34 comprises two 
versatile finite impulse response filter ("VFIR") chips 
?*?■?■'.. 3412. Each VFIR chip embodies large scale 
integration and incorporates at least one adder tree 
circuit 3415 and a plurality of multiply-accumulator 
(T- or "MAC") circuits 3416. The convolution unit 34 
additionally comprises delay lines 3413, 3414 and a 
morphological and convolution control (MORCON) unit 
3410, depicted in greater detail in FIG. 6F. 

Referring to fig. 6F, in response to synchroniza- 
tion signals (s.) from CRX 31, the MORCON unit 3410 by 
means of its components ("Part 1-), i.e., delay lines 
3420, 3421, 3422, 3423 and Sync CONVOL unit 3424 pro- 
vides synchronization signals for the logic unit 35 and 
for the delay lines in CONVOL unit 34; by means of Sync 
MORPH unit 3425 ("Part 2"), the MORCON unit 3410 
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provides synchronization signals for the delay lines 
incorporated in MQRPH unit 36. 

The convolution unit 34 can be configured 
(FIG. 6A) to compute, as a 22-bit output on AA_Bus, the 
sum of the two 16-bit inputs on A_Bus and on B_Bus, 
each input first multiplied by as constant/ i.e., 
AA « aA + jffB. In this computation, the output on 
BB_Bus la The delay circuits 3413, 3414 have no 

part in this computation. Therefore, CTq, CTj signals 
which control the operation of the delay lines 3413, 
3 414 are low during this computation (FIG. 6N) . Only 
delay line 3420 (Delay 0) of the MORCON unit (FIG. 6F) 
la used in thia computation. 

The convolution unit 34 can be configured 
(FIG. 6B) to compute, as a 2 2 -bit output on AA_Bus, the 
product of the two 16-bit inputs on A_Bus and B_Bus, 
i.e., AA » A X B. The delay circuits 3413, ^ 3414 have 
no part in thia computation. Therefore, CTq^ CTj 
aignala are low during this computation (FIG. 6N) . 
Only delay line 3420 (Delay 0) of the MORCOM unit (FIG. 
6F) is uaed in this computation- 

The convolution unit 34 can be configured 
(FIG. 6C) to compute a convolution defined - by a 
kernel of size up to 3 x 4 - on up to three lines of up 
to 512 pixels each, e.g., lines to L^,,, Mo to Mjn, and 
No to Mj,,, where each pixel has 16-bits. In this 
computation, delay line 3421 (Delay 1) of the MORCON 
unit (FIG. 6F) is employed. CTq and CT, signals look 
like a 2 -bit binary counter (FIG. 6M) . In this 
configuration the MAC circuits 3 416 are coupled in twos 
and compute a sum of two elementary products, which are 
finally added to form the result of the 3x4 kernel 
convolution, at the rate of HPIX clock (FIGS. 6M, 6N) . 
For example, with reference to FIG. 6M, the partial 
result R(M1) for lines - Ly, M, - M,, - N,, is given 
by the following equation: 



R(K1) - C^Lj 

With reference to FIGS, 6C, 6M, the two right-most 
MAC circuit iirj in VFIP. 34:2 are fed with the current 
'"line" of pikeis via del^^y line 3 413. The two left- 
most MAC circuits 3416 of VFIR 3412 are fed with the 
following line of pixels (line + 1) of the image and 
the two central MAC circuits 3415 in VFIR 3411 are fed 
with the previous line of pix^sls (line - 1) via delay 
line 3414 . 

At the rising edge of HPIX clock, three new pixels 
one per lij^ne enter the CONVOL unit. At the same time, 
current pixels are shifted out (FIG. 6H) of each MAC 
circuits 3416 in order to satisfy the computational 
algorithm. 

The convolution unit 3 4 can be configured 
(FIG, 6D) to compute a convolution of a I x 16 kernel. 
The delay lines 3413, 3414 have no part in this 
computation. CTq and CT, signals look like a 2-bit' 
binary counter (FIG. 6M''). In this configuration the 
MAC circuits 3416 are chained to compute a sum of 16 
elementary products at the rate of HPIX clock. 

In the convolution of a 1 x 16 kernel, the 
CT signals impart two states, State 0 and State 1, to 
each of the MAC circuits 3416 in VFIR chips 3411, 3412 
(^IGS. 6D, 6H) . Thus, as each input pixel, P0-P5U 
(FIG. 6H) , is shifted through a MAC circuit 3416, two 
computations are made in each MAC circuit 3416, one 
during State 0 and the second during State 1. At the 
end of the cycle, the eight parti-ii results are added 
no forra the result of the 1 x 16 Kernel convolution. 

30 
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Wrien thr first pixel, Pq, has been shifted through both 
VFIR chips 3411, 3412 and appears at the pixel output 
of the fourth MAC circuit 3416 of VFIR chip 3411, pixel 
Pi6 is entering the first MAC circuit :f4l6 or VFIR chip , 
3412 (FIG. 6H) . For example, with reference to FIG. 
6M", the partial result for pixels - P,,, i.e., R(P8) , 
is given by tha following equation: 
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The convolution unit 34 can be configured (FIG. 
6E) to compute a convolution of a 1 x 32 kernel. The 
delay lines 3413, 3414 have no part in this computa- 
tion- Signal CTo is asserted high at a 2a Mhz rate and 
signal CT, is asserted high at a 10 Mhz rate, thereby 
producing four states (FIGS. 6M'", 6N) , i.e.. States 

0, 1^ 2 , and 3 . 

In this computation, the eight MAC circui-s 3416 
are chained to compute a sum of 32 elementary products 
at the rate of HPIX clock. Thus, as each input pixel, 
Pd-Psh (FIG. 61), is shifted through a MAC circuit 3416, 
four computations are made in each circuit 3416, one 
during State 0, the second during State 1, the third 
during State 2, and the last during State 3. At the 
end of the cycle, the eight partial results are added 
to form the result of the 1 x 32 kernel convolution. 
Wnen pixel P^ has be«n shifted to the pixel output of 
VFIR chip 3411 (FIG. 61), pixel P33 enters the pixel 
input of VFIR chip 3412, For example, with reference 
to FIG. 6M''', the partial result for pixels - Pjj, 

1. e., P(P16), is given by the following equation: 
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The CTo, CT, signals are generated by State Machine 
3526 (FIG. 6F) under control of a 40 Mhz clock, whereby 
CTo is always low or else has a frequency of 20 MHz 
and CT, is always low or else a frequency of 10 MHz 
(FIG. 6N) . In sununary, the CT signals define States of 
Operation of the VFIR chips 3 411, 3412 in computations 
of 3 X 4, 1 X 16 and 1 x 32 kernels (FIGS. 6M, 6N) . 

Each MAC. circuit 3416 (FIG. 6J) comprises four 
data registers 3440, four coefficient value registers 
3441, and seme multiplexers (MUX) 3442 to feed a 
multiply-accumulator (T COEFF. x DATA") block. The 
MUX on the DATA input allows data to come from the 
SHIFT input when the 3416 circuits are chained. 

In general, coefficient values are some constants 
loaded during the programmation of the 3 411, 3412 VFIR 
chips . 

Each VFIR 3411 and 3412 comprises an internal 
sequencer (FIG. 6J) which synchronizes the computation. 
The sequencer is externally controlled by CTo and CT, 
signals. Appropriate data and coefficient values are 
stored in registers 3440, 3441 (FIG, SJ) . Under the 
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control of CTq^ CTi signals, the sequencer computas and 
establishes the required electrical paths from the 
registers 3 440, 3 441 to the multiply-accumulator 
COEFF. X DATA") block via multiplexers 3442. 
The delay lines 3413, 3414 (FIG- 6) operate like 
RAM memory; delay is a function of how many elapsed 
HPIX periods there are between writing to the delay 
lino (input) and reading from the delay line (output) 
{FIGS. 6K, 6L) • Two signals, a clock signal (CLK^, 
CLKt) and a reset signal (R,w/ R^r) / perform the 
necessary timing (FIGS. 6K, 6L) . Reset signals (R^w/ 
^) are asserted at the same time to produce one line 
pt delay between the input and the output of each delay 
line 3413, 3414. Reset signals are for initialization 
pf the delay lines and are computed from the input 
synchronization signals S| transmitted from CRX 31 
(FIG. 6). 

E. Logic Unit 3 5 

The logic unit 351 ( FIG. 7) provides miscellaneoua 
logic functions with respect to 22-bit input images 
from convolution unit 34 via AA_Bus and 16-bit input 
images from unit 3 4 via BB_Bus, A delay line 359 
orovides the necessary delay to get the 17-bit output 
of shifter 356 in phase with the 16-bit BB_Bus input, 
as both are applied to the logic unit 351. The logic 
functions with respect to the input images are 
performed under control of initialized fimction 
register 352 and synchronization unit 362. These 
functions' Include AND, OR, exclusive OR~(XOR) , NOT A, 
Shift A (0 to 6 bits) , and calculations of maximum and 
minimum values (MIN/MAX) for the most significant bit* 
{••MSB") of the 17-bit output of shifter 356, 

The image shifting function is performed by 
shiftar 356 (FIG. 7) which can be visualized as 
inclur'ing a shiftable 17-bit "window" 363 (FIG. 7A) 
overlying a 22-bit regirtiar 364, The purpose of the 



Shifter 355 i.v uo enpib'jn scaling of the dcV^a contained 
in the 22-bit xmeae signal received on the /vA_Eus from 
convolution unit 34. In rhe in;: tiaiizetion of logic 
unit 35, r.he windosv -^3 is -}^t to tnv far left with ths 
17 window hits (0-.;.5 p.us D?.L. 17) overlying the left- 
mosr 17 register oLtsi ■'5-21), This represents the 
default setting of "O", The rightwarc . shift of the 
window 3 63 can be procraioined during operation from 0 to 
6 bits of shift to obtain ths highest precision for the 
result (i.e., the greatest number of significant bits). 

It should be notsd that the 2 2 -bit output on 
AA_Bus from CONVOL unit 34 is a number truncated from a 
larger number cf bits. In accordance with the present 
invention, in order to irvprove significantly the 
precision of subsequent calculations performed by MORPH 
unit 3 6 upon image data transmitted from the logic unit 
35 via the BAL_Bus (16 bits) and the BBL_Bus (16 bits), 
the 17th bit transiaitted via the BAL 17 line from the 
shifter 356 to MORPK unit 36 is in the nature of a 
carry bit for the next summation step. Thus, although 
the 16-bit BAI._Bus output represents truncation of 
the 22-bit data input on AA_Bus, the combined 17-bit 
BAL_Bus and BAL 17 output is not a mere truncation of 
the 22-bit data input on the AA_Bus. Rather, the 
combined 17-bit BAL_Bus and BAL 17 output data can be 
used in the unit 3-:^ at the us..r's option, as a 

rounding off at tha 22-bit /^._Bus input data, the 1-bit 
BAJL 17 acting a carry for tne n-:t suraaation steo. 

^^MIN/KLAX det-ctt^r 357 of logic unit 25 (FIG. 7)" 
<'na^u.-& coniput:i^t;;.on of rhe minimum and ma::iir.um values 

r .• _^'^s. The purpose of the 
co.u-.ur.c-r.ion 'CO .:.void lot^s of precision in 

the co."\putat onn o^-.-,-.. - ... 

I . -..--.j oy -onvoiution unic 34, 



b?'-rn tr'jncatfed to 22 bits 



•^•'V-'-^il .'.s rounded cif t:o 15 bits 
pj us 3. 17-:-. c ■ .-- ■ • -.- ■■ - 



RG^crring to FIG= MlN/hu-X G^;tGc::or 357 

comprises HAX det^^ctor .5 571, MI-' di^t^^tor 357 2, MAX 
regisi:e^ 3573, MIN regisner 35"s-, ^nu It ipiexer 

fMUx; 3 575. The GT input tc r^qic^ter 3 5-; 3 is 

reprsser-' *^ ve cf whether the nev;iy coir.rjuted iriaximura is 
greater ^ :--f. tt^e current ^p.aximuir. value. SiTuilarly, the 
LT input to MIN register 3 57 4 is representative of 
whether the newly computed minitnuir;: is less; than the 
current minimujR value. 

In operation f the coinpu^iation cf MIN/KAX values is 
useful to scale the coefficient values used in the 
operation of the VFIR chipr^ 3411, 3412 (FIG. 6) when 
performing convolution functions. The result computed 
by the COiTVOL unit 3 4 is a sum of products in the form 
of Wj • I^, where the weighting factors are chosen to 
a\'oid overflow. If the computed MAX value is too small 
(i.e., not enough significent bits) the Wj can be 
adjusted to increase the precision of the result of the 
convolut.^ on, A similar operation can be done with the 
MIN value when the W; hcve to be decreased. The MIN/MAX 
values are stored in two eight bit registers, i.e., MAX 
REG 3573 and MIN REG 3574 (FIG. 7B) , which can be read 
by the control processor 21 at specific addresses on 
the secondary Bus SDATA (FIG. 7) . 

Mixing unit 353 {FIG, 7) and the initialised 
mixing register 3 54 provide, where required, bit 
mapping operations where image data from the BS_Bu3 can 
be mapped into and stored in data transmitted via the 
AA_E>us. The mapping operation provides a means to save 
storage space when working on binary imeiaj^z. An eight 
b'.c wide image can store eight different pieces of 
binary Information, that is, bit map information. 
Delay line 3eo provides appropriate dslay for che 
B3_BuG dat.-i being applied tc the r..i::inQ unit 353. 
in ciccordance with the invention, fcr furt;ier 
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signed/unsigned computation, the MS bit of the BAL_Bus 
can be inverted by the mixing unit 353. 

For some functions to ba calculated by the MORPH 
unit 36, all bits of BB^Busdata can be forced to zero 
by prograituaed operation of the zeroing unit 3 61. 

Synchronization of logic unit 35 is provided by 
synchronization unit 3 62 under the control of synchro- 
nization signals (sj transmitted from CONVOL unit 34. 
Output synchronization signals (s^,) from sync unit -362 
are transmitted to MORPH unit 36. 

F. Morphological Unit 3 6 

Referring to FIGS. 8A and 83, the morphological 
unit 36 is a high speed computation circuit for 
computing numerical values and binary vailues on gray 
scale electronic images and on binary electronic 
images* The unit 36 comprises two processors 361, 362 
for performing most of the algorithms used in 
mathematical morphology, a plurality of delay lines 
363, and a second plurality of multiplexers 364 for 
operating the chips 361, 362 in serial mode or in 
parallel mode. 

Each chip 361,362 is a Mathematical Morphology 
Integrated Processor ("PIMMl") supporting the latest 
algorithms of mathematical morphology. 

The PIMMl chip has a programmable internal 
architecture. The PIMMl chip can perform morphological 
and point-to-point transf ormc^tions on binary and 8-bit 
ne images xn hexagonal and square grids. 

The point processing unit of the PIMMl chip can be 
used to threshold, add, subtract and compare numerical 
images, to perform boolean operations such as AND, OR, 
an'i Exclusive Or (XOR) between binary images, and to 
Lnyarr binary images. 

The PI:lmi chip also includes a programmable 
aorphr Logical processing unit which works on a 3 x 3 
<Qrnal, This processing unit can ba configured as 

3> G 



1818^2 • 

eight binary processors associated in pipe-line or in 
parallel; each of the eight processors can perform 
binary morphological transforms such as Hit or Miss 
Transform (H.H.T-) and thinning and thickening. Two of 
the eight processors can perform recursive binary 
reconstruction and first point extraction. The 
programmable morphological processing unit also can be 
configured as two numerical morphological processors 
allowing numerical transformations such as dilation, 
erosion, thinning, thickening, gradient and arrowing. 
In addition, said processing unit can be configured as 
recursive hexagonal, square (4 or 8 connectivity) and 
dodecagonal distance function processors. 

The PIMMl chip also includes a measurement unit 
which is able to compute binary area or numerical 
volume upon images up to 4096 x 4096 pixels wide. 

The full capability to associate several PIMMl 
chips in pipe-line architecture enables a substantial 
increase in the performance of the digital image 
processing circuit 3 0 of the present invention. 

Moreover, \:wo PIMMl chips associated in parallel 
allow 16-bit gray-tone image processing. 

The configuration of the PIMMl can be done by the 
programming of 34 internal 8-bit registers. 

In the parallel mode of operation of chips 361, 
362 (FIG. 8A) , the LSB bits of the BAL_Bu3 and BBL_Bua 
are processed in PIMM chip 3 61 to output eight LSB bits 
on the BAP_Bus, The serial/parallel mode multiplexers 
364 are appropriately enabled both to couple synchro- 
nization signals to PIMM chip 3 62 and also to couple 
the MSB bits on the BAL^Bus and BBL^Bus to PIMM 362, 
which outputs aighx: MSB bits on the BAP_Bus, 
simultaneously with the output of eight LSB bits on the 
BAP_Bus by PIMM .131. 

In the serial mode of operarion of chips 361, 362 
(FIG. 83) , Che LSB bits are compuced by chip 361 and 



the result :,b l;r;5n5;7;.lt :;f^d to input A cf r.-.lp 262. 
together with a '=~arry" s Lg'-';il to input F of chip 3 62 
"•ia apprcpriat-:;: j y <^-nable;j ir.ui tip] exers 364. Chip 362 
continues the ccr;puti:ti.an which ra?"ults in the MSB 
output on DAr_Bus. Syncr.rcni -at ion signals are trana- 
mitted to chip 362 via chip "ei and an appropriately 
enabled multiplexer 364. Chip 362 also transisits the 
synchroniza-cion signals? to L'JT unit 37, 

A novel mechanisia ha^ beer? iir.plement:ed for the 
CONVOL unit 3 4 to coffiput^i Images vhose borders are 
greater than one pi>:?.] in v-idth. In accordance with 
the present invention, the input synchronization sig- 
nals are mads compatible with the PIKMl chip specifica 
tions (i.e. one border pixel) and the synchronization 
signals issued from the MORPH unit 3 6 are rebuilt to 
restore the original border. This is done in the 
synchro unit 3 62 of LOGIC unit and in the synchro unit 
378 of the LUT block 37, 

Look-Un Tahle On it 3 7 
As noted elsewhere herein, the digital image 
processing circuit 30 (FIG. 2B) constructed and 
operated in accordance with the present invention 
comprises a pipeline procesni.ng or computation 
(computing) chain (indicated generally by reference 
numeral 379 in FIGS. 2B, 9F-9I? composed of the 
convolution unit 34, the logic unit 35, and the 
morphological unit 3 5 whose output is transmitted on 
BAP_Bu8 to the Chain's last component, i.e., look-up 
table (LUT) 37 (FIG. 2B) , 

In accoraance vith th^ present invention, the 
look-up tabic, urit iV i5 constructed and ooerated to 
provide xook-up runct:ic.as for both gray scale 

electronic .....er, i . . , .^nu.ncricai LUT" functions, and 

also for b i r =^ ^ V r? - t- . 

-^.r.^.y c^.-_c.: ^ iir.aces, i.e., ^'binary LUT- 

fu net: ions. Wh^rJir • t-^^. v 

•5»-'-i.c.. ^..e .ina.if. being processed by the 

^ J. ^ - . a rj^rziY -ca^e image or a 



binary- 'r^^g^, when tafi Ic-s:*'. pl*^"^l 1:5 tra^ns-irted on 
K_Bus from th& look-up tsbis- 37 and vrirten into Image 
Itfil! 32.. the computational functions perfox:iiie;ci by the 
chcin which resu3tr-:d ir; r.hat outpi;.t, are c^ripieted. 

Look-up table unit 37 compr is^^s (TIG. 9) look-up \ 
table circuitry 37i cind iooK-ur trj-.ble PJ\M nernorv 372, I 
Circuitry 371 comprises a mode register 373.. an address'' 
generator 377 for loading addre^,se-> for L(JT RAH 372, a 
plurality of nsultiplexsrs 375, a second plurality of 
load circuits 376, a mumory bank (-'Page") register 374, 
and a synchronization unit 378. 

The RAM 3 72 is arrayed in tv^o sections, MSB and 
LSB (FIG. 3) , each section being S bits t-'id^i. LlIT data 
is stored in VRArl Bank 0 (FIG. lA) . i-men the load-LUT 
commands are run, the unit 37 is initialized and LUT 
data are written to RAM 372, under the tiining control 
of synchronization unit 378 (FIG. 9A) , and ^.re stored 
therein at addresses generated on the fly by address 
generator 375, when the LUT data are at the input of 
RAM 372 (FIG, 9M) • 

During initialization (FIGS. 9A, 9F) , the LUT data 
are transmitted from CRX 31 (FIG, 9F) through that part 
of the processing chain 379 comprising units o4, 35, 
and 36 to the LUT unit 37 via BAP_Bus , Under timing 
control provided by synchronization unit 378, the LUT 
data are transmitted to RAM 372 from BAP_Bus via the 
dashed line paths (FIG. 9A) through enabled load 
circuits 376. During initialization of LUT unit 37, 
thti units 34, 35, and J 6 are rendered transparent to 
LUT data from CRX 31 (FIG. 9F) . 

As noted elsewhere herein, the LUT KAxi 372 can 
contain numarical LJT functions and bAnary LUT func- 
tions. Ihe LUT RAM 37 2 orgc nization i,^ usar dafin^.d 
aMd cc-n be organized as two numerical LL^T furictions 
.n.^.nx O, Brink i;, as 512 binary LOT functions (Bank 0, 
3ar.k 1), or es a ra: xed drran.^e„;>- r- 1 wizh ore n^.nerical 
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LUT function (Bank O) and 256 binary LUT functions 
(3ank 1) (FIG. 9L) . During processing of an image, the 
internal address generator 377 is in standby. LUT RAM 
addresses are provided by pixels on BAP-Bus, 

In processing 16- bit numerical images (FIGS. 9B, 
9G, 9H) , the look-up table 37 pnrforms a translation 
process with respect to each 16-bit input pixel (Pq,) 
appearing on the BAP_Bu3, i.e., the l6-blt output pixel 
(Po'jt) appearing on R_Bus is some function of the input 
pixel: pQin- - f (?»,)# where the function f is stored in 
the numerical LUT portion of RAM 372 and can be 
"identity"; sc[uaring; square root; exponential e; log 
to any base; etc. 

In processing 8 -bit binary images one at-a-time 
(FIGS. 9C, 91), the least significant bits ("LSB") chip " 
361 and most significant bits ("MSB") chip 36^ are 
operated in serial mode (FIG. 83) . The significant 
image data in an 8 -bit binary image are in the MSB bits 
of BAP_Bus. With respect to the way the RAM LUT has 
been initialized (FIG. 9M) , the MSB of BAP_Bus must be 
put on the LSB address of the RAM LUT (FIG. 9L) . 

In operation, the 8-bit MSB input on BAP_Bus is 
used via enabled multiplexers 375 (dash-dot path in 
FIG. 9C from BAP_Bus to RAM 372) to generate an 
address, the Bank (Page) register 374 identifying the 
starting memory location for the binary LUT (FIG. 9L) ; 
the computed a-bit output from RAM 372 is transmitted 
pn the LSB bits of R Bus. 

In processing 8-bit binary images two at-a-time 
(FIGS. 90, 9J) , the LSB and MSB chips 361, 362 are 
operated in parallel mode (FIG. 8A) . In this opera- 
tion, the LUT block architecture allows the performing 
pt a LUT function for the LSB information while 
bypassing the MSB information directly onto the MSB 
part of R-bua... 

I/O 
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In operation, tha two 8 -bit binary images appear 
on tha MSB and LS3 bits of BAP_Bus and are used via 
enabled multiplexers 375 (dash-dot path in FIG, 9D from 
BAP_Bus to RAM 372) to generate an address. Bank 
(Page) register 374 identifies the top of the binary 
^LUT in RAM 372 (FIG. Mode register 373 under 

synchronization signal control enables the multiplexers 
375 in the path from BAP_Bus to RAM 372 (FIG. 90) and 
tha multiplexer 375 in the path for the MSB bits to be 
transmitted from RAM 372 to R_Bus (dashed line in 
FIG. 9D) . At the same time, the LSB bits are 
transmitted from RAM 372 to R_Bus bypassing the output 
terminal of disabled load circuit 376. 

FIGS. 9E, 9K depict the operation of LUT unit 37 
to perform an "identity" function on a 16-bit numerical 
image by using a binary identity LUT. An "identity" 
function is a function that imposes no change on tha 
input, i.e., the output of LUT unit 37 is equal to or 
the same as the input to LUT unit 37. 

In this operation, for a 16-bit numerical image, 
tha MSB bits on BAP_Bus are transmitted directly 
through LUT circuit 371 via two enabled multiplexers 
375 to the MSB of R_Bus (dashed line MSB path from 
BAP_Bus to R_Bua in FIG. 9E) . Additionally, tha LSB 
bits on BAP_Bua, together with the signals generated by 
Mode and Page (Bank) registers 373, 374, described 
elsewhere herein (dash-dot path in FIG. 9E) , are used 
to generate, an address for reading the LSB bits from 
RAM 372 and transmitting those LSB bits to the LSB line 
on R_3us (dashed path to R_Bus LSB in FIG. 9E) . 

A3 described elsewhere herein, tha synchronization 
unit 378 (FIG. 9) 3ynchronirea tha input to the look-up 
table 37 and the storage of LUT data in RAM 372 and 
transmits synchronizarion signals (sj to tha address 
generator. 
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H. Components of Preferred Embudi Tq^nt 



The specific components incorporated in the pre- 
ferred embodiment of the subsystem 20, including the 
digital image processing circuitry 30, constructed in 
accordance with the present invention and described 
with reference to FIG. 2B can be constructed from 
discrote elements or advantageously from integrated 
circuits. The following table lists examples of such " 
components . 
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All logic units doocribod with reference to the drawings contain 
st-^ndard TTL-Faat and CMOS PAL coniponents- 
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Thus, in accordance with the presant invention, 
novel methods for digital signal filtering' and for 
general purpose digital image processing and novel 
apparatus, for performing digital signal filtering and 
for perfcmlng general purpose digital image processing 
have been described in detail above. 

♦ 

While specific embodiment 3 of the invention have 
been disclosed, variations in procedural and structural 
detail within the scope of the appended claims are pos- 
sible and are contemplated. 

For example, as described elsewhere above, 
although MIN/MAX computations are made on the eight 
most significant bits, such computations can be made 
on the full range of le bits. Additionally, the pro- 
cessing circuit is not limited to processing 512 x 512 
X 16 bit physical images, but can be used to process 
1024 X 1024 X 16 bit physical Images, and larger. 

Additionally, by addition of random access memory 
chips to LUT RAM 372, RAM 372 can be enlarged to three 
or more Banks as desired by the user. Moreover, image 
storage RAM 32 can be enlarged by the addition of 
random access memory chips, e.g., to provide memory for 
ptoring intermediate results computed by pipeline 
processing chain 379. 

Moreover, the subsystem 20 (FIG. 2A) is not 
limited to use with a host computer 10 embodying MCA 
architecture. By means of an interface circuit 14 
suitably designed according to the specifications of 
non-MCA architecture host computer and the subsystem 
20, the subsystem 20 can be connected to and be 
operated with a host computer embodying any other 
architecture, e.g., rjsc, EISA, etc. 

Further, logic unit 35 performs a number of 
Miscellaneous functions described above, to which 
addxtxonal functions can be added. Additionally, to 




increapQ the spaed of computations, one or mora 
additional pipeline processing chains 379 (FIG. 2B) can 
be added to the processing circuit. In this embodi- 
ment, the 16-bit R_Bus output of LUT unit 37 in the 
first chain 379 can be connected to the A_Bus input of 
the CONVOL unit comprising the first stage of the 
second pipeline processing chain 379, with the R_Bu3 
output of the LUT unit in the second chain connected to 
the crj( 31 via R_Bus or alternatively to the A__Bus 
input of the third pipeline chain, etc. in that 
manner, any desired number of pipeline processing 
chains 379 can be connected between the 16-bit R__Bus 
output of tUT unit 37 in the first processing chain 379 
and the R_Bus input to the CRX 31 FIG. 2B) . 

There" is, therefore, no intention of limitation to " 
the Abstract or to the precise disclostire herein 
presented. 
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K Digital image processing ci^-cuinry (30) comprising cross 

bar.'^unlt (31), image storage ram (32), address generator (^3), 
coHTolution unit^ (34) logic unit (35), morphological unit (36) 
and lc3iok-u^ table uult (37) 

2. The processing circuitry (30) as claimed in claim 1, 

wherein said cross bar unit (31) comprises synchronization unit 
C315). 

3* The processing circuitry. (30) as claimed in preceding 

claims vhereln said address gensrator (33) comprises ALU control 
(333), X scan counter (334) and Y scan counter (336), XL register 
(335). YL register (337), ALUX unit (331), ALUY unit (332)(U^ 
write unit (341) 

The processing circuitry (30) as claimed in preceding 
clairafc wherein said convolution unit (34) comprises VFIR chips 
(3411, 3412). 

5' The processing circuitry (30) as claimed in preceding 

clalBs wherein said logic unit (35) comprises function register 
(352), synchronization unit (362), shifter (356), MIN/MAX 
detector (357)^ mixing unit (353), 

^' The processing circuitry (30) as claimed in preceding, 

claims wherein said morphological unit (36) comprises two 
processors (361 ^36 2) 

^' The processing circuitry (30) as claimed in preceding 

claims wherein said look-up table unit (37) comprises ram memory 
(372) and look-up table circuitry (371) 

Dated this 2v';th day of Nay 1994, 
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